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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

5,809,471 Brodsky 9-1998 

IBM VoiceType Dictation Wayback Machine Archive: 4-1 997 

http://yvww.voicerecoqnition.com/ibm/ 
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ibm31des.html 

6,489,979 Belknap et al. 12-2002 

5,500,919 Luther 3-1996 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

In regards to claim 19, Brodsky discloses a method for searching in a multimedia signal 
(see -), wherein the multimedia signal includes a first data format 

1. Claims 19-32, 35-36 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Brodsky (US Pat. 5,809,471). 

In regard to claim 19, Brodsky discloses a method for searching in a 
multimedia signal (see Brodsky: column 1, lines 50-56), wherein the multimedia 
signal includes at least a first data format component (extracted component such 
as closed captioned text) and a second data format component (retrieved 
additional information; see Brodsky: column 1, lines 66-67, column 2, lines 20-23, 
column 3, lines 17-21; the additional information or program relevant information 
maybe provided by the broadcaster, see column 6, lines 44-48), the method 
comprising the steps of: 

Receiving a search parameter (see Brodsky: column 2, lines 5-9); 
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Analyzing the first data format component of the multimedia signal to 
identify the occurrences of the search parameter (the step of matching a search 
request to the extracted information components requires the step of analyzing 
the extracted information or the first component in order to find the match; see 
column 2, lines 25-27, lines 30-32, lines 55-61); and 

For at least one occurrence of the search parameter in the first data 
component (i.e. the extracted data), presenting a corresponding second data 
format segment (i.e. the retrieved additional data that is provided by the 
broadcaster and presented to user; see column 6, lines 44-48 and column 3, 
lines 33-36) of the multimedia signal. 

In regards to claim 20, the first data format component is a closed caption 
component (see column 2, lines 20-23, column 5, lines 36-37), and wherein the 
second data format is an audio component (hearing additional retrieved 
information that is of further interest to user; see column 5, lines 11-14, column 6, 
lines 13-15). 

In regard to claim 21, Brodsky discloses maintaining a dynamically 
changing dictionary that determines what a valid searchable keyword is. The 
dictionary only contains keywords or items extracted from most recently received 
portion of the program. See column 4, lines 1-3. Therefore a user can only 
search for a particular keyword within a predetermined period of time (i.e. 
keyword in the recently received portion of program) after the occurrence of the 
keyword (i.e. search parameter) in the closed caption component. When a user 
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enters a valid searchable keyword, the occurrence of the keyword is matched in 
the dictionary first and then a corresponding additional retrieved information is 
presented to the user. Since a user can only search recently occurred keywords 
that are stored in the dictionary, the corresponding additional retrieved 
information (in this case audio data that user is interested in hearing) begins and 
ends within a predetermined period of time before and after the occurrence of the 
search parameter in the extracted data (i.e. closed caption component). 
Consider the following example. If a news program is broadcasted containing a 
segment about France presented, and the dictionary only extracts keywords 
occurred in the last 10 minutes, the additional retrieved information regarding 
France maybe retrieved and presented only within 10 minutes of the occurrence 
of the word France. As a result, Brodsky anticipates the step of the 
corresponding second data format segment being a section of the audio 
component that begins and ends within a predetermined period of time before 
and after the occurrence of the search parameter in the closed caption 
component. 

In regards to claim 22, the first data format component is a closed caption 
component (see column 2, lines 20-23, column 5, lines 36-37) and wherein the 
second data format component is a video component (see column 5 lines 28-30, 
column 6, lines 13-15). 

In regards to claim 23, see claim 21 . For similar reasons as stated for 
claim 21 above, Brodsky anticipates the step of the corresponding second data 
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format segment being a section of the video component that begins and ends 
within a predetermined period of time before and after the occurrence of the 
search parameter in the closed caption component. 

In regards to claim 24, Brodsky discloses as an example presenting a map 
of France as the additional retrieved information in response to user's request of 
France (see column 6, lines 39-42). Such a display of map anticipates the scope 
presenting a "still image" from the video component (map of France provided by 
the broadcaster, see column 6, lines 44-48) that is presented at the occurrence 
of the search parameter in the closed caption component. 

In regards to claim 25, the first data format component is an audio 
component and the second data format component is a video component (see 
column 4, lines 37-44, column 5, lines 29-31). 

In regards to claim 26, the formats of the first data format component and 
the second data format component are selected from the group consisting of: text 
data; closed caption data; audio data; speech data; video data (see column 4, 
lines 19-23). 

In regards to claim 27, Brodsky discloses the step of receiving search 
parameters in the form of spoken words, voice format, etc. See column 6, lines 
24-26. Brodsky further discloses that the words are extracted from closed 
caption text and stored in the dictionary. See column 2 lines 20-23. Brodsky 
also discloses that the recognition processor maybe similar to the 'IBM Voice 
type Dictation Solution" that converts voice dictation to text. See column 4, lines 
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64- column 5 line 3. Therefore Brodsky anticipates the step of receiving a search 
parameter in a third data format (voice/audio format), converting the search 
parameter from third format to the first format (i.e. text) via recognition processor 
such as IBM Voice Type Dictation Solution. 

In regards to claim 28, the format of the third data format component 
(speech) and the first data format component (closed caption text) are selected 
from the group consisting of: text data; closed caption data; audio data; speech 
data; video data; (see column 2 lines 20-23 and column 4 lines 64 - column 5 
line 3) 

The limitations of claim 29 are anticipated by claim 19 and are analyzed 
as above. Furthermore, the retrieving the additional information (see column 6, 
lines 44-48) anticipates the step of identifying a corresponding segment of a 
second data format component in the multimedia signal. 

In regards to claim 30, the second data format component is a video 
component (see column 4, lines 19-23); the method further comprising the step 
of displaying the segment of the video component to the user (see column 3, 
lines 34-37); 

In regards to claim 31 , see claim 24 above. In the above example, when 
the additional information such as the map of France is presented to the user, the 
map of France anticipates a "single image" being presented to the user. 

In regards to claim 32, see claim 24 above. 
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In regards to claim 35, the first data format component is a text component 
(see column 2, lines 20-23, column 5, lines 36-37) and the method further 
comprises the steps of: 

Receiving the search parameter in an audio format (see column 6, lines 24-26); 
and 

Converting the search parameter to a text data format prior to analyzing the 
multimedia signal (see column 4, lines 62-67 and column 5, lines 1-3). 

In regards to claim 36, the voice recognition performs the converting step 
converting step using a speech to text converter since the first data format is a 
text component (see column 5, lines 37-38) and the search parameter is in audio 
format (see column 6, lines 24-26). 

2. Claims 33-34 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Brodsky (US Pat. 5,809,471). 

In regard to claim 33, Brodsky discloses that the first data format 
component (extracted components) is an audio component (see column 4, lines 
37-44) and further comprises various forms of search request, including selection 
from a visual menu of dictionary words (see column 5, lines 14-20, column 6, 
lines 23-26). 

Brodsky does not disclose whether the selectable menu comprise items 
(searchable parameters) in text format or other icon format. 
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Examiner takes official notice that it was well known in the art at the time 
of the invention to present a visual menu of selectable keywords in text format 
that a user can read and select. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the system by presenting a selectable menu of search 
keywords in a text format that the user can read and select; and further be 
obvious to converting the search parameter (selected keyword) to the data 
format of the first data (i.e. audio data) so that the selection parameter can be 
matched with the first data component thereby allowing the retrieval of the 
second data component. The motivation for converting the search parameter 
from a third format to the first format is for ease of processing the data for 
comparison rather than converting an entire dictionary into another format for 
comparison (i.e. it is easier to convert a word or two into a second format to 
compare with the dictionary, rather than convert the entire dictionary into the 
format type of the search keyword for comparison). 

In regards to claim 34, the modified system above converts the text data 
of the selected keyword into audio data using a text to speech converter. 
(10) Response to Argument 

Claim 19 recites, "a method for searching in a multimedia signal...". In 
response to appellant's arguments on page 4 regarding claim 19, the recitation 
"searching in a multimedia signal" has not been given patentable weight because 
the recitation occurs in the preamble. A preamble is generally not accorded any 
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patentable weight where it merely recites the purpose of a process or the 
intended use of a structure, and where the body of the claim does not depend on 
the preamble for completeness but, instead, the process steps or structural 
limitations are able to stand alone. See In re Hirao, 535 F.2d 67, 190 USPQ 15 
(CCPA 1976) and Kropa v. Robie, 187 F.2d 150, 152, 88 USPQ 478, 481 (CCPA 
1951 ). While appellant attempts to refute the Brodsky reference stating that 
there are no teachings in Brodsky of searching in a TV or telephonic input signals 
(see Appeal Brief pages 4, 7) the examiner notes there are no steps in the body 
of claims 1 9-28 that depend on the step of "searching in a multimedia signal". In 
regards to appellants arguments on page 5, Examiner also notes that claim 19 
fails to recite that a first and second data components originate from the same 
source. However, to address appellant's arguments stating that Brodsky fails to 
support this feature, the examiner to draw appellants attention to column 6, lines 
44-48 in Brodsky. Brodsky discloses presenting a second data format (i.e. 
retrieved additional information) of the multimedia signal, wherein the retrieved 
additional signal maybe provided by the broadcaster and therefore part of the 
same signal source (i.e. signal source provided by broadcaster) the extracted 
components (e.g. closed captions) were extracted from. Examiner further notes 
that "a multimedia signal" has a broader scope than what applicant contends and 
can alternatively interpreted as the multimedia signal presented to the user on 
display 108 of Brodsky the multimedia signal comprising a first component 
(signal from the TV input signal) and a second component (retrieved information). 
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See figure 1 of Brodsky. Appellants continues to argue in page 5 that, "Brodsky's 
vocabulary is a dictionary made up of information extracted from multimedia 
signal by content extractor 102 and stored in a buffer 104" and "as such, 
Brodsky's vocabulary is not a component of a multimedia signal as required by 
claim 19". The examiner respectfully disagrees. Since the dynamically changing 
dictionary is made up of words constantly extracted from the TV feed, the 
extracted words (first component) are thus included in the multimedia signal (i.e. 
TV signal) in order to be extracted from the multimedia signal. In further regards 
to appellants arguments on claim 19 in page 6, when such an occurrence of a 
search parameter is identified in the first component (extracted words included in 
the multimedia signal), additional retrieved information is presented to the user, 
wherein the additional retrieved information (second component) is provided by 
the broadcaster (see column 6, lines 44-48) and hence the multimedia signal 
(signal provided by broadcaster) also includes the second component (additional 
retrieved information). 

Regarding applicant's arguments of claims 33 and 34, references 
supporting official notice taken have been provided. In particular, Belknap et al. 
(US Pat. 6,489,979) discloses a system comprising a menu of selectable 
(textual) keywords that a user can select from to access additional information. 
See column 14, lines 1-7 and figures 5A-6E. Luther (US Pat. 5,500,919) further 
discloses the use of a text to speech converter for converting input text 
information to speech data. See abstract. 
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(1 1 ) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 

For the above reasons, it is believed that the rejections should be sustained. 
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Unleash the Power! 

IBM® has concentrated on two very important factors in building their speech recognition product. This is SPEED and ACCURACY. 

Available in Windows 3.1, Windows '95 and OS/2 systems, IBM Voicetype also comes in several different languages, and it also has 
industry specific vocabularies for Journalism, Legal and Medical entities. 

Add-Ons include Kolvox OfficeTALK & Kolvox LawTALK. 



Voice-to-text capability 

• Facilitates the creation of documents 

• Saves time and increases productivity by eliminating transcription 

• Provides a solution for people who are unable (or unwilling) to use a keyboard 

Hands-free, eyes-free operation 

• Enables users in technical, production and office environments to dictate text, product reports or fill in forms while using their 
hands and eyes for other tasks 

• Physicians and other medical personnel can read x-rays or examine patients and describe findings simultaneously 

Large General-language vocabulary 

• Provides a ready-to-use language model designed to serve a wide range of professional and business users 

• Helps ensure accurate word capture 

• Allows you to customize the system easily by adding up to 2,000 words to the vocabulary 
Optional industry-specific vocabularies 

• Ready-to-use solutions for radiology, emergency medicine, journalism and legal enhance applications 

• Specialized terrninology and word-pattern models enable the system to frequently recognize words that are run together, slurred 
or partially obscured by background noise 

• Support for additional industry-specific vocabularies is continually being expanded 

Speaker enrollment function 

• Maximizes dictation accuracy by creating a model for every individual speaker 
Clipboard data transfer 

• Transfer dictated text to OS/2 and Windows applications by cut-and-paste, using voice, keyboard or mouse commands 

• Use keyboard emulation to transfer data into applications that do not support cut-and-paste 

Navigation controls 
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IBM Voicetype Dictation 



http://web.archive.org/\veb/19970415092814/http://\v\vw.voicerecog.. 



• Use voice commands to operate OS/2, Windows and compatible applications 

• Includes automatic "speech enabling" of the OS/2 Workplace Shell and Presentation Manager controls, as well as Windows 
icons and tool bars 

Voice Action Editor 

• Allows users and developers to define macros that perform a sequence of actions with a single voice command 

• Facilitates development of customized, application-specific solutions 

• Increases productivity by letting users insert prerecorded phrases or paragraphs of repetitive text with a single spoken word 
PCMCIA VoiceType Dictation Adapter 

• Supports notebook computers for transportable dictation solutions 

• Helps make it easy for people to use the system, wherever they need to work 

Multiple language support 

• Available for OS/2 and Windows in U.S. English, U.K. English, French, German, Italian and Spanish. 



& Minimum 486/SX/SLC 25 MHz (recommended 486DX2/66) 
& 16 MB of RAM (recommended 24 MB) 

& 33 MB dedicated hard disk space | additional 30 MB when enrolling 
& OS/2 Ver. 2. 1 or higher for OS/2 configuration 
& Microsoft Windows 3 . lx or Windows 95 



Pricing 



~ ORDER 




ONLINE- 



NOW 



~ Consultation Request Form ~ 
Download the IBM Voicetype Demonstration 



More Questions?? Contact us by email, snailmail, telephone or fax. 

You may e-mail tech support 

2 1 st Century Eloquence 
205 Worth Avenue, #201 
Palm Beach, Florida 33480 

Telephone: 800-245-2133 Fax: 407-835-4901 



Copyright © 1996 21st Century Eloquence, Inc. 

Microsoft is a registered trademark and Windows is a trademark of Microsoft Corporation® (tm) 
Other brand and product names are trademarks or registered trademarks of their respective holders, 

AH comments welcome 

If you have trouble with any of our pages, please contact wwwmasterfypboLcom 
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